Labels

Veeam - Job has failed unexpectedly

Environment
  • Veeam Backup 5.x

Content
We have set Veeam Backup to notify us what the status of backup operation is every day; however, it didn't send a few time, because of an error.
Job has failed unexpectedly

As tried the solution from Veeam did not solve this problem, we went to look for more detail from the event logs. It showed that might be caused by the database. Therefore, we set the scheduled scanning of the anti-virus application excluding the database files.
C:\Program Files (x86)\Microsoft SQL Server\MSSQL.1\*.*

C:\Program Files (x86)\Microsoft SQL Server\MSSQL.2\*.*
However, those similar event logs had been continuously there.
Type,Date,Time,Source,Category,Event,User,Computer
Information,11/24/2012,3:17:34 AM,SceCli,None,1704,N/A,BackupServer
Error,11/24/2012,3:00:46 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,3:00:46 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,2:04:23 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,2:04:23 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,2:04:23 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,2:04:20 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,1:31:59 AM,SQLBrowser,None,8,N/A,BackupServer
Error,11/24/2012,1:31:59 AM,SQLBrowser,None,8,N/A,BackupServer
Information,11/24/2012,1:06:52 AM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/24/2012,1:02:52 AM,MSSQL$SQLEXP_VIM,(2),833,N/A,BackupServer
Information,11/24/2012,12:59:38 AM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/24/2012,12:52:50 AM,MSSQL$SQLEXP_VIM,(2),833,N/A,BackupServer
Information,11/24/2012,12:52:51 AM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/24/2012,12:18:22 AM,MSSQL$SQLEXP_VIM,(2),833,N/A,BackupServer
Information,11/24/2012,12:18:23 AM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/24/2012,12:09:28 AM,MSSQL$SQLEXP_VIM,(2),833,N/A,BackupServer
Information,11/24/2012,12:00:28 AM,crypt32,None,2,N/A,BackupServer
Information,11/24/2012,12:00:27 AM,crypt32,None,7,N/A,BackupServer
Information,11/23/2012,11:52:08 PM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/23/2012,11:27:09 PM,MSSQL$SQLEXP_VIM,(2),833,N/A,BackupServer
Information,11/23/2012,11:27:09 PM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/23/2012,11:08:16 PM,MSSQL$VEEAM,(2),833,N/A,BackupServer
Information,11/23/2012,11:08:16 PM,MSSQL$VEEAM,(2),17883,N/A,BackupServer
Information,11/23/2012,11:08:15 PM,MSSQL$SQLEXP_VIM,(2),833,N/A,BackupServer
In that time, we thought, "Is it a hardware question." It was because our SMART monitor notified that.
BackupServer (1) ST31000528AS... Degradation: (C4) Reallocation Event Count [1882->1889]
It would be a question of bad sectors, - http://kb.acronis.com/content/9132.
9132: S.M.A.R.T. Attribute: Reallocation Event Count, “Reallocation Event Count S.M.A.R.T. parameter indicates a count of remap operations (transferring data from a bad sector to a special reserved disk area - spare area).”
To ensure that was because of the hard disk, suggested from this site, we scheduled monitoring at the same time of backup operations. The result was not normal.
Average = 0.105, minimum = 0.002, maximum = 1.090
Avg. Disk sec/Transfer
- ideal value: 0.005 - 0.010
- if >= 0.015, bottleneck(any delay)

Performance > New log settings... > Counters
- Use local computer counters
- Performance object: PhysicalDisk
- Counter: Avg. Disk sec/Transfer
- Instances: the hard disk wanted


Thus, we called HP to replace the hard disk, then backup operations have been back to normal.


Reference
  • \Administrator\Local Settings\Application Data\Veeam\Backup\VeeBackupService.log
    [23.11.2012 23:08:20] <11> Error at Veeam.Backup.DBManager.CDbAccessor.ExecNonQuery(String spName, SqlParameter[] spParams)
    [23.11.2012 23:08:27] <14> Info Opening client session...
    ...

    ...
    [23.11.2012 23:39:57] <11> Error at Veeam.Backup.DBManager.CDbAccessor.ExecNonQuery(String spName, SqlParameter[] spParams)
    [23.11.2012 23:59:46] <10> Info There is no new deferred tasks for process.
    ...

    ...
    [24.11.2012 01:07:03] <11> Error at Veeam.Backup.Service.CServiceWorkingThread.DoRetryLoop()
    [24.11.2012 01:07:08] <11> Info Backup manager starting dispatch loop.
    ...

    ...
    [24.11.2012 02:01:14] <11> Info SureBackup isn't enabled in this license edition, skip sure backups jobs processing.
    [24.11.2012 02:04:24] <08> Error Job [ServerOne] cannot be started. SessionId: [bab4c714-c996-44c4-8a7c-63a658b6cc93], Timeout: [197 sec]


Update
Labels: