CN101707543B - Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type - Google Patents
Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type Download PDFInfo
- Publication number
- CN101707543B CN101707543B CN2009103107033A CN200910310703A CN101707543B CN 101707543 B CN101707543 B CN 101707543B CN 2009103107033 A CN2009103107033 A CN 2009103107033A CN 200910310703 A CN200910310703 A CN 200910310703A CN 101707543 B CN101707543 B CN 101707543B
- Authority
- CN
- China
- Prior art keywords
- task
- scheduling unit
- task scheduling
- path
- executing units
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 230000005540 biological transmission Effects 0.000 claims abstract description 39
- 238000012544 monitoring process Methods 0.000 claims abstract description 11
- 238000005194 fractionation Methods 0.000 claims description 27
- 238000012546 transfer Methods 0.000 claims description 15
- 230000005059 dormancy Effects 0.000 claims description 9
- 230000000284 resting effect Effects 0.000 claims description 9
- 239000012634 fragment Substances 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 230000008878 coupling Effects 0.000 claims description 4
- 238000010168 coupling process Methods 0.000 claims description 4
- 238000005859 coupling reaction Methods 0.000 claims description 4
- 230000009471 action Effects 0.000 claims description 3
- 230000013011 mating Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000003672 processing method Methods 0.000 abstract description 2
- 238000012552 review Methods 0.000 abstract 1
- 230000009466 transformation Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 230000006872 improvement Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000009931 harmful effect Effects 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 206010033799 Paralysis Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to an enterprise media bus system supporting multi-task type and an enterprise media bus method supporting multi-task type, namely an electronic computer network system and a data processing method. The system comprises a client, a buffer memory device, a database, a task scheduling unit, a plurality of task executing units, an application and interface server, a configuring tool and a BS system monitoring server, wherein the database, the task scheduling unit, the plurality of task executing units, the application and interface server, the configuring tool and the BS system monitoring server are connected with one another and are connected with the client through Ethernet networks. Because of the adoption of the task scheduling unit and the plurality of task executing units, the system supports various types of tasks, and supports on-line transmission, transcoding, technology review, computation and verification of MD5, and hybrid tasks of the subtasks. By splitting a task and detecting the service condition of a hardware system, the enterprise media bus system makes full use of the read and write bandwidth of a disk and CPU resource according to configurable task combination,, thereby greatly improving overall system performance.
Description
Technical field
The present invention relates to a kind of enterprise's media bus system and method for supporting the multitask type; Being a kind of electronical computer network system and data processing method, is that a kind of enterprise that is used for TV station or other making and use digital television program is used for inner network system and the method that transmits the Television Digital media file.
Background technology
For an enterprise that has a large amount of digital media files between inner all departments, to transmit mutually, the network system that is specifically designed to the digital document transmission of big capacity, high bandwidth is very necessary.Digital media file is not a simple carrying problem in the transmission of enterprises, is the complex process that needs provide various functions.With the TV station is example, and a TV station has a plurality of audio frequency related service systems of looking, and each operation system is inner, and major part all can have demands such as transcoding, the skill of looking audio file are examined, transmission.When looking the audio file exchange between the operation system, more left not the above-mentioned functions demand, enterprise inner or other making and use digital television program of a therefore common TV station all can set up enterprise's media bus.
The media bus EMB of enterprise (Enterprise Media Bus) is that the bus architecture of audio file transmissions function is looked in a kind of main realization.Domestic and international existing EMB system or type EMB system are not very complete on systemic-function at present.Present existing EMB system has only supports transformation task, two kinds of tasks of support transcoding that has and transmission.Existing EMB system does not support the mixed type task that independent skill is examined, independent MD5 calculates checking and multiple subtask (as: transcoding+skill is examined+MD5, transmission+skill is careful+MD5 etc.).It is the abbreviation of technological audit that skill is examined.It is the objective evaluating to the picture signal technical quality of program that traditional skill is examined, and mainly refers to the test of indexs such as picture carrier, R-Y, B-Y amplitude and colour pase; Along with going deep into of TV station's digital improvement; The business such as adopting, compile, broadcast, manage, deposit of TV station has got into digital Age; Program and material also exist with the form of digital file mostly, examine except traditional skill is examined content to the skill of digital file, also the compressed digital of file are looked audio content and verify; Comprise: the checking of the checking of file structure aspect, compression and encoding context, file are looked the checking of audio content, thereby seem more complicated.Existing EMB system does not support to comprise in the task situation that transcoding, transmission, skill such as examine at a plurality of independent subtasks yet, needs to submit to separately for a plurality of dissimilar subtasks like this, uses very inconvenient.If the file destination that the source file that the machine that has in the EMB system can only access task, the machine that has can only access tasks, but do not have the machine source and target file of access task simultaneously, for this situation, existing at present EMB system can not handle.A physical file possibly have multiple different access mode: Ethernet file-sharing mode, optical fiber access mode, FTP access mode, the priority of different access mode is different.When a kind of access mode occurs when wrong, the user hopes that the EMB system can select other access modes automatically.Existing EMB system does not support that the path load under this complex network environment is balanced.Along with the high resolution video and audio applied more and more, the file volume is increasing, and file format becomes increasingly complex various, the EMB task is carried out efficient have higher requirement, and task executions efficient need improve in the existing EMB system.The disk read-write bandwidth of high-performance storage networking can be very high, when carrying out transformation task in the EMB system, if the transmission speed of task is not limited, may take very high readwrite bandwidth.Total bandwidth is limited, and the operable bandwidth of other operation systems will significantly reduce like this, thereby influences the normal use of whole service system.Existing EMB system need realize transformation task speed limit function according to the demand of using bandwidth, to guarantee the EMB system bandwidth is reasonably taken.
Summary of the invention
For solving prior art problems, the objective of the invention is to propose a kind of enterprise's media bus system and method for supporting the multitask type.Said system and method is realized supporting the multitask type, and is made the present invention make full use of resource through intelligentized design through the fractionation to task, carries out various tasks fast and efficiently.
The objective of the invention is to realize like this: a kind of media bus system of enterprise that supports the multitask type; The media bus system of enterprise of described support multitask type comprises: task is submitted client, buffer storage, database, task scheduling unit, a plurality of task executing units, application and interface server, configuration tool, BS system monitoring server to; Described database, task scheduling unit, a plurality of task executing units, application and interface server, configuration tool, BS system monitoring server via Ethernet interconnect and submit to client to be connected with task, and described task executing units is connected with the memory device of each system that need carry out the digital media file exchange.
Use a kind of enterprise's media transmission method of supporting the multitask type of said system, the step of said method is following:
Add the step of data transfer task: be used for task and submit to client to propose data transfer task, and the task scheduling unit is added data transfer task through application and interface server;
Data transfer task is analyzed the step that splits with one-level: be used for the task scheduling unit transfer of data task definition and demand are analyzed; Task definition comprises: independent transcoding, transmission, skill are examined the MD5 checking, or mix multiple subtask: transcoding adds that skill is examined, transcoding adds skill and examines and add MD5, transcoding and add MD5, transmission and add that transcoding, transmission add MD5, transmission adds transcoding and adds MD5; To the task that mix multiple subtask, the one-level that system carries out task splits, and is split as the subtask of a plurality of single task role types automatically;
The step that task scheduling unit task scheduling begins: after being used for the success initialization of task scheduling unit, the beginning duty cycle;
Whether detect the step whether idle task executing units is arranged: whether be used for the detection of task scheduling unit has the task of idle task executing units and each task executing units full; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step;
The step whether waiting task is arranged: be used for the task scheduling unit and detect whether waiting task is arranged; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step;
The balanced step that detects of path load: be used for the task scheduling unit task can be detected in employable each path, the parameter of detection comprises: path priority level, speed-limiting messages;
Judge whether the step in operable path: be used for the task scheduling unit and use the requirement in path and the operating position of existing route to compare to task; Judged whether operable path; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step;
Task is carried out the step that secondary splits: be used for the task scheduling unit task or the subtask that the one-level task splits the single task role type that obtains split; Comprise: FTP splits, middle route splits, distributed fractionation, so that according to the actual performance element and the path of the state assignment task of path and task executing units;
The step of coupling path and task executing units: be used for the access ability of task scheduling unit, select to be fit to carry out through splitting the various piece path and the task executing units of task according to task executing units and path;
The step of Processing tasks: be used for task executing units task is handled, in the task implementation, according to the progress report time interval and the percentage-proportion intervals of configuration, to task scheduling unit Report Tasks progress;
The step of updating task state: be used for the information that the task scheduling unit receives that the Task Progress report updating task of task executing units is carried out;
Task is carried out the step of end and result's readjustment: accomplish in the task executing units task and carry out the back to task scheduling unit Report Tasks execution result; The task scheduling unit is through the task result readjustment service of application and interface server calling task submission side, readjustment task action result;
Judge whether the step that manually stops: be used to judge whether manually stop the task scheduling circulation, if " denying " then get back to " step that task scheduling unit task scheduling begins ", if " being " then stop task scheduling;
Query task result's step: be used for the task side of submission to calling application and interface server; Query task implementation progress and execution result; This step and " judge whether manually stop step " parallel execution, and can carry out in each process in " step of Processing tasks " and carry out.
The beneficial effect that the present invention produces is: owing to be provided with buffer storage, database, task scheduling unit, a plurality of task executing units; The present invention supports polytype task; Be supported in that transmission, transcoding, skill are examined, calculating and the checking of MD5; And the hybrid task of these tasks (transcoding+skill is examined+MD5, and transmission+skill is careful+MD5 etc.).The present invention splits and the path dynamic load equilibrium technology through dynamic task, and the task under the complex network environment is handled has good support.The present invention makes up according to configurable task, supports the dissimilar a plurality of tasks of the concurrent execution of unit, has made full use of disk read-write bandwidth and cpu resource, and overall system performance is improved a lot.The present invention with the distributed executive mode of single transcoding task multimachine device, makes full use of the clustering functionality of system through the fractionation task, has improved the efficient of transcoding task greatly.The present invention supports transformation task speed limit function through the configuration of detection dish battle array and memory block speed limit, make the present invention to the bandwidth of operation system in controlled range, avoided high bandwidth to take the harmful effect that operation system is produced.
Description of drawings
Below in conjunction with accompanying drawing and embodiment the present invention is described further.
Fig. 1 is the embodiment of the invention one said a kind of enterprise's media bus system principle diagram of supporting the multitask type;
Fig. 2 is the embodiment of the invention five said a kind of flow charts of supporting enterprise's media transmission method of multitask type;
Fig. 3 carries out the sub-process figure in the step that secondary splits to task in the said a kind of enterprise's media transmission method of supporting the multitask type of the embodiment of the invention six.
Embodiment
Embodiment one:
Present embodiment is a kind of media bus system of enterprise that supports the multitask type, and is as shown in Figure 1.The media bus system of enterprise of the described support multitask of present embodiment type comprises: task is submitted client, buffer storage, database, task scheduling unit, a plurality of task executing units, application and interface server, configuration tool, BS system monitoring server to; Described database, task scheduling unit, a plurality of task executing units, application and interface server, configuration tool, BS system monitoring server via Ethernet interconnect and submit to client to be connected with task, and described task executing units is connected with the memory device of each system that need carry out the digital media file exchange.
Enterprise's media bus (Enterprise Media Bus, the English EMB that is called for short) system is a kind of based on bus architecture, concentrates looking the system that audio frequency is handled.The characteristics of bus architecture are to be responsible for the Unified Treatment to other each operation system data by a centring system, rather than handle through the direct exchange of each operation system.Under the bus architecture; Each operation system only exchanges with centring system, passes between each system mutually, and newly-increased or deletion operation system does not have influence to other system; Therefore bus architecture has reduced the coupling between the operation system, has simplified the deployment and the maintenance of system.Present embodiment is a bus architecture; Couple together each operation system; Unification is handled looking audio frequency media; Accomplish media file mutual between each operation system and transmit, the main processing mode comprises the transmission of looking audio file, the transcoding between the different file format, and automatically skill examine, MD5 verification etc.MD5 is a kind of disclosed data summarization algorithm, and through this algorithm, the file of each different content can obtain a different MD5 value.If file has any modification, the MD5 value all can change.Therefore, whether whether the MD5 verification is can authenticating documents complete, be modified.
The effect of the said system of present embodiment each several part:
EMB database: mission bit stream, task executing units and the task scheduling unit information of storage whole system, and system parameters configuration.
Task scheduling unit (Manager): be responsible for task is carried out dynamic resolution and scheduling.The task scheduling unit can be arranged on and form the task scheduling server in the separate server, or shares a server with task executing units.An independent task scheduling server both can be set in an EMB system, the backup server of task scheduling server also can be set outside the task dispatch server, two or more task scheduling servers form heat and are equipped with, and realize the high availability of system.The task scheduling unit further is embedded in the task executing units, can simplified system dispose, and improves Fault Tolerance.Present embodiment is an intelligent system, can confirm the process of transmission according to the ruuning situation of system, utilizes resource to greatest extent, efficiently executes the task with the fastest speed.The task scheduling unit plays critical effect in the present embodiment; The path of the continuous detected transmission media file in task scheduling unit, the running status of storage system; Through analysis and fractionation to task, find best task execution path, it is efficient that realization is executed the task.
Task executing units (Actor): task executing units is connected with the memory device of each system that need carry out the digital media file exchange; Be responsible for carrying out the EMB task; Read the source media file of task; As required to goal systems write media file, carry out that automatic skill is examined, transcoding or MD5 verification, and Report Tasks progress and result.
Use and the external interface server: dispose the WebService interface module of the said system of present embodiment, and the monitoring of BS is used.The WebService interface module is deployed in the service invocation request that the task of external system submits to client to submit to except acceptance division, and present embodiment is also realized the loopback of task result through this module.
Configuration tool: information such as responsible operational factor to system, store path, task executing units, task scheduling unit are configured.
The BS system monitoring: present embodiment provides the BS monitoring based on Web, and the user can check the running status of EMB system through browser.BS is the abbreviation of Browser Server structure, is based on a kind of system design deployment and the operational mode of browser server framework.Supervisory control system is the BS framework, and APD is deployed on the application server, and the BS monitor client only need just can be visited through browser, no longer other softwares need be installed.This software architecture has been simplified the installment work of BS monitor client, is convenient to software upgrading, and is also user-friendly, and supports through Internet remote access.
Buffer storage:, do not have buffer storage can't realize that then middle route splits and FTP splits for route fractionation and FTP fractionation task in the middle of carrying out provide buffer memory.
Embodiment two:
Present embodiment is the improvement of embodiment one, is that embodiment one is about task scheduling unit and task executing units refinement.Present embodiment described task scheduling unit and a plurality of task executing units are separately positioned in the server separately, form task scheduling server and a plurality of task and carry out server.
Task scheduling unit and task executing units all are arranged in the server separately in the present embodiment, that is to say to arrange a task executing units, perhaps a task scheduling unit in the server.The communication that task is carried out between server and the task scheduling server can be adopted other modes such as MQ, Socket.The advantage of present embodiment setting is that hardware requirement is low, but disposes complicated.
Embodiment three:
Present embodiment is the improvement of embodiment two, is the refinement of embodiment two about the task scheduling server.The described task scheduling server of present embodiment has subsequent use task scheduling server, is connected through heartbeat between described task scheduling server and the subsequent use task scheduling server.
Because the key effect of task scheduling unit, and the task scheduling unit is arranged on separately in the station server, and in a single day this server goes wrong; For example crash; Paralysis will appear in whole business data transmission system, and therefore for improving fault-tolerant ability, present embodiment is provided with task scheduling server standby host; A task scheduling unit promptly is set in another station server again, forms task scheduling server host and task scheduling server standby host.The hardware of task scheduling server standby host and task scheduling server host, software systems dispose just the same, have only increased switching device shifter.
Task scheduling server standby host and task scheduling server host Update Information synchronously, when switching, just can not occur being interrupted of any task execution like this.Task scheduling server host and task scheduling server standby host are a kind of addresses; Promptly be called the task scheduling server host at just at work task scheduling server; The task scheduling server that is in SBR is called task scheduling server standby host; After switching, task scheduling server standby host originally becomes the task scheduling server host, and task scheduling server host originally becomes task scheduling server standby host.
Detect through heartbeat between task scheduling server host and the task scheduling server standby host, whether inspection task scheduling server host survives.If the task scheduling server host occurs unexpected, task scheduling server standby host is through switching oneself being updated to the task scheduling server host.
For increasing reliability; Present embodiment can also be provided with many task scheduling server standby hosts; On each task scheduling server standby host a task scheduling unit is arranged respectively; These task scheduling server standby hosts realize having only a task scheduling server standby host can be updated to the task scheduling server host through the database locking mechanism.
Embodiment four:
Present embodiment is the improvement of embodiment one, is the refinement of embodiment one about the task scheduling unit.The described task scheduling of present embodiment unit is embedded in the task executing units.
The task scheduling unit of present embodiment is embedded in the task executing units, existing task executing units in the server so just occurs, and the situation of task scheduling unit is also arranged.This mode no longer need be disposed independently task scheduling server, has reduced the system deployment complexity.Owing to can embed the task schedulings unit, that is to say to have occurred that the task scheduling unit is arranged in a plurality of servers that standby host each other between the task scheduling unit has strengthened fault-tolerant ability in each server in a plurality of task executing units.
After in a plurality of task executing units, embedding the task scheduling unit, a plurality of task schedulings unit occurs, one of them can be set be the main task scheduling unit.After the server at place is made mistakes, other any one have a task scheduling unit server can work on as subsequent use task scheduling unit, strengthened Fault Tolerance.
Embodiment five:
Present embodiment is to use a kind of enterprise's media transmission method of supporting the multitask type of the said system of the foregoing description, and flow process is as shown in Figure 2.
Present embodiment is through the mode of tree-shaped task structure, task fractionation, task plug-in unit; Support transcoding, transmission, independent skill to examine, independent MD5 calculates and the hybrid task of the task of types such as checking and these subtasks (transcoding+skill is examined+MD5, transmission+skill is careful+MD5 etc.).And can also support more task type through the mode of increase task plug-in unit.
Dynamically carry out because task splits, adopt the task tree structure, can realize the task dynamic resolution easily, tree is also very directly perceived on displaying interface, is convenient to the user and understands.Specify the task tree structure below: each task all has a root node; Essential informations such as the title of description task, submission person, submission time; Specifying informations such as the source file of task, file destination are described in child node, have only the afterbody node (leaf node) in the task tree to be only finally can directly carrying out of task.For example, the user has submitted a hybrid task that comprises transmission and two task types of transcoding simultaneously to, can generate a task root node and transmission, two task child nodes of transcoding automatically in the system.System is when handling transformation task; Possibly find that this transformation task need carry out the centre route and split; So for the transformation task child node, can be split as the subtask of two next stage automatically: source path is to the subtask of buffer storage, and buffer storage is to the subtask of destination path.When system handles the transcoding task; Possibly find that need carry out FTP to this task splits; So for transcoding task child node, can be split as the subtask of two next stage automatically: FTP is to the copy subtask of middle buffer memory from the source, and middle buffer memory is transcoded onto the destination path subtask.(if destination path also is FTP, can be split as three subtasks).In the present embodiment, the structure of final task tree is following:
The task root node
The transmission subtask
Source path is to the subtask of buffer storage
Buffer storage is to the subtask of destination path
The transcoding subtask
Source FTP is to the copy subtask of middle buffer memory
Middle buffer memory is transcoded onto the destination path subtask
Present embodiment can adopt tree-shaped task structure, in the task implementation, according to system configuration, task is split, and dynamically generates new subtask node, then new subtask is handled.Present embodiment has disposed the multiple path access mode of each physical storage area, comprises Ethernet file-sharing mode, optical fiber access mode, FTP access mode etc., and every kind of access mode can be provided with priority, maximum number of connections.Which task executing units can also be set visit which path, so just can know the corresponding relation between each path and each task executing units.
If the file destination that the source file that the server that has in the system can only access task, the server that has can only access tasks, but there is not the station server source and target file of access task simultaneously.For having disposed buffer storage in this situation present embodiment; Present embodiment is split as the source to this task to buffer storage automatically; Carry out the subtask that buffer storage is carried out to two orders of target; Thereby split out two subtasks nodes to an original task node, route split in the middle of this task dynamic resolution was called.
A physical file possibly have multiple different access mode: Ethernet file-sharing mode, optical fiber access mode, FTP access mode, the priority of different access mode is different.When a kind of access mode occurs when wrong, the user hopes that the EMB system can select other access modes automatically.Existing EMB system does not support that the path load under this complex network environment is balanced.To this situation, present embodiment is provided with the task scheduling unit.The task scheduling unit is when carrying out task scheduling; According to the corresponding relation between path priority, the current linking number in path, path maximum number of connections, path and the task executing units, calculate optimum path and execute the task with the task executing units of mating most.Increase the current path linking number before executing the task, the current path linking number is reduced in the back of executing the task, and has realized the dynamic load leveling in path like this.
Special, for the transcoding task, after the path load equilibrium, when all available paths all comprise the FTP path, do not support the form of the direct transcoding of FTP for some, the task scheduling unit can carry out FTP task dynamic resolution.FTP task dynamic resolution is divided into following three kinds of situation:
If just the source is FTP, can be split as task from the source and copies buffer storage+buffer storage to target transcoding two subtasks.
If just target is FTP, can be split as task and be transcoded onto buffer storage+buffer storage from the source and copies target two subtasks to.
If source and target all is FTP, can be split as task and copies buffer storage+buffer storage from the source to and be transcoded onto buffer storage+buffer storage and copy target three subtasks to.
Present embodiment can be supported the dissimilar a plurality of tasks of the concurrent execution of unit according to configurable task combination, has made full use of disk read-write bandwidth and cpu resource, and overall system performance is improved a lot.The task of general transport-type can take a large amount of disk read-write bandwidth, but only takies very little cpu resource.Transcoding type task is just opposite, can take a large amount of cpu resources, but few for bandwidth occupancy.Present embodiment can make up task type according to configuration, lets transport-type task and transcoding type task on a machine, carry out simultaneously, can farthest utilize disk read-write bandwidth and cpu resource like this, improves the whole efficiency of system.
Present embodiment can make full use of the clustering functionality of a plurality of task executing units through the distributed execution technique of single transcoding task multimachine device because a plurality of task executing units are arranged, and has improved the efficient of transcoding task greatly.
Because the transcoding task needs the long period usually, for the very high transcoding task of some priority, the user hopes to finish the work as early as possible, and present embodiment then uses a plurality of task executing units this task of executed in parallel simultaneously, and the transcoding task is accomplished rapidly.
Present embodiment splits the realization distributed trans-coding through the distribution of task.Task is carried out distributed fractionation; Each transcoding task is carried out by many task executing units servers simultaneously; Carry out the automatic merging of file destination in the implementation, this mode has made full use of the task executing units clustering functionality, has improved transcoding task executions efficient greatly.
Present embodiment is supported transformation task speed limit function through dish battle array and the configuration of memory block speed limit, make present embodiment to the bandwidth of operation system in controlled range, avoided high bandwidth to take harmful effect to the operation system generation.
Present embodiment is in operation and detects the spendable bandwidth in path, in the hope of formulating best task implementation strategy according to existing bandwidth resources.On the memory block, increased the notion of dish battle array.Dish battle array and memory block are the relations of one-to-many.Can on the dish battle array, total speed limit bandwidth be set, acquiescence single task maximum bandwidth, three parameters of acquiescence single task minimum bandwidth.In the memory block below the dish battle array,, single task maximum bandwidth and single task minimum bandwidth can be set also in each path owing to a memory block possibly have the multiple path of different access mode.The task scheduling unit of present embodiment is when executing the task scheduling; Except carrying out the path dynamic load leveling according to path priority and linking number; Also according to the maximum speed limit parameter of coiling battle array and path, and when shroud battle array utilized bandwidth, calculate the maximum available bandwidth of this task.If this task maximum available bandwidth is less than the task minimum bandwidth that is provided with, this task wouldn't be carried out, wait other inter-related tasks complete after, when satisfying bandwidth speed limit condition, carry out this task again.The utilized bandwidth of increase dish battle array before carrying out, the utilized bandwidth of minimizing dish battle array after carrying out.Through this speed limit mechanism, make present embodiment to the bandwidth occupancy of operation system in controlled range, avoided high bandwidth to take the harmful effect that operation system is produced.
The step of the said method of present embodiment is described below:
Add the step of data transfer task: be used for task and submit to client to propose data transfer task, and the task scheduling unit is added data transfer task through application and interface server.Present embodiment adds task interface and can be set to support tree-shaped task interface, uses tree-shaped task interface can realize complicated task execution sequence.
Data transfer task is analyzed the step that splits with one-level: be used for the task scheduling unit transfer of data task definition and demand are analyzed; Task definition comprises: independent transcoding, transmission, skill are examined the MD5 checking, or mix multiple subtask: transcoding adds that skill is examined, transcoding adds skill and examines and add MD5, transcoding and add MD5, transmission and add that transcoding, transmission add MD5, transmission adds transcoding and adds MD5.The basic task type of present embodiment support comprises that transcoding, transmission, independent skill are examined, independent MD5 calculates checking etc.Also to support the task of multiple subtask type hybrid, examine+MD5 like transcoding+skill, transmission+skill is careful+MD5 etc.Because the complexity of task; Simple processing can't be carried out the task of such complicacy, therefore at first must analyze task, if task has comprised a plurality of task types simultaneously; The one-level of then carrying out task splits, and is split as task the subtask of a plurality of single task role types.
The step that task scheduling unit task scheduling begins: after being used for the success initialization of task scheduling unit, the beginning duty cycle.
Whether detect the step whether idle task executing units is arranged: whether be used for the detection of task scheduling unit has the task of idle task executing units and each task executing units full; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step; Being not only that whole task executing units is idle in this step just thinks the free time when the task of task executing units is full inadequately, to think that also this task executing units is idle.
The step whether waiting task is arranged: be used for the task scheduling unit and detect whether waiting task is arranged; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step.
The balanced step that detects of path load: be used for the task scheduling unit task can be detected in employable each path, the parameter of detection comprises: path priority level, speed-limiting messages.
Judge whether the step in operable path: be used for the task scheduling unit and use the requirement in path and the operating position of existing route to compare to task; Judged whether operable path; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step.
Task is carried out the step that secondary splits: be used for the task scheduling unit secondary fractionation is carried out in task or subtask that the one-level task splits the single task role type that obtains; Comprise: FTP splits, middle route splits, distributed fractionation, so that according to the actual performance element and the path of the state assignment task of path and task executing units.
The step of coupling path and task executing units: be used for the access ability of task scheduling unit, select to be fit to carry out through splitting the various piece path and the task executing units of task according to task executing units and path.
The step of Processing tasks: be used for task executing units task is handled, in the task implementation, according to the progress report time interval and the percentage-proportion intervals of configuration, to task scheduling unit Report Tasks progress.
The step of updating task state: be used for the information that the task scheduling unit receives that the Task Progress report updating task of task executing units is carried out, so that the task scheduling unit is other Task Distribution resources.
Task is carried out the step of end and result's readjustment: accomplish in the task executing units task and carry out the back to task scheduling unit Report Tasks execution result; The task scheduling unit is through the task result readjustment service of application and interface server calling task submission side, readjustment task action result.
Judge whether the step that manually stops: be used to judge whether manually stop the task scheduling circulation, if " denying " then get back to " step that task scheduling unit task scheduling begins ", if " being " then stop task scheduling.
Query task result's step: be used for the task side of submission to calling application and interface server; Query task implementation progress and execution result; This step and " judge whether manually stop step " parallel execution, and can carry out in each process in " step of Processing tasks " and carry out.
Embodiment six:
Present embodiment is the improvement of embodiment five, be embodiment five about task is carried out the refinement of secondary splitting step, flow chart is as shown in Figure 3.
Present embodiment is said, and that task is carried out the substep that step that secondary splits comprises is following:
Need to judge whether the substep of FTP fractionation: be used for judging whether the transcoding task must comprise the FTP path; And determine whether needs carry out FTP and split according to looking audio format configuration; If " be " then get into " substep that FTP splits ", if " deny " then get into " substep of route fractionation in the middle of whether needing ".
The substep that FTP splits: be used to utilize buffer storage; Whether according to source or target is the FTP path, is split as 2 or 3 subtasks to a FTP transcoding task: FTP is to the copy of middle buffer memory from the source, middle buffer memory transcoding; Behind the transcoding to the copy of target FTP, updating task tree structure simultaneously.
Whether the substep whether in the middle of needing route splits: being used to check has the task scheduling unit of access originator and destination path simultaneously, if not then need carry out centre route fractionation.Route splits then and gets into " substep that middle route splits " if desired, if route splits then gets into " whether the distributed fractionation substep of needs " in the middle of not needing.
The substep that middle route splits: utilize buffer storage, be split as two subtasks to a task: source path subtask in the buffer storage, buffer storage is to the subtask of destination path.
The substep that whether needs distributed fractionation: for the transcoding task; Option during according to the task of submission to; Task priority, the task executing units quantity factor of current free time, whether decision carries out distributed fractionation; If " be " then get into " substep of distributed fractionation ", if " deny " then get into " step of mating path and task executing units ".
The substep of distributed fractionation:, tear a transcoding task for a plurality of transcodings subtask open and to add an automatic merging task according to the quantity of present idle task performance element; There is a fragment sequence number each transcoding subtask; Be responsible for generating a fragment of file destination,, merge task automatically by the concurrent execution of a plurality of task executing units; Can in time merge into the final objective file to the fragment of transcoding completion according to the fragment numeric order.
What should explain at last is, more than only unrestricted in order to technical scheme of the present invention to be described, although the present invention is specified with reference to the preferred arrangement scheme; Those of ordinary skill in the art is to be understood that; Can be to technical scheme of the present invention (such as the bus form that system uses, the configuration of each service, connected mode; The front and back sequential scheduling of each step) makes amendment or be equal to replacement, and do not break away from the spirit and the scope of technical scheme of the present invention.
Claims (5)
1. enterprise's media transmission method based on the support multitask type of the media bus system of enterprise that supports the multitask type; Described bus system comprises: task is submitted client, buffer storage, database, task scheduling unit, a plurality of task executing units, application and interface server, configuration tool, BS system monitoring server to; Described database, task scheduling unit, a plurality of task executing units, application and interface server, configuration tool, BS system monitoring server via Ethernet interconnect and submit to client to be connected with task; Described task executing units is connected with the memory device of each operation system that need carry out the digital media file exchange, it is characterized in that the step of said method is following:
Add the step of data transfer task: be used for task and submit to client to propose data transfer task, and the task scheduling unit is added data transfer task through application and interface server;
Data transfer task is analyzed the step that splits with one-level: be used for the task scheduling unit transfer of data task definition and demand are analyzed; Task definition comprises: independent transcoding, transmission, skill are examined, the MD5 checking, or mix multiple subtask: transcoding adds that skill is examined, transcoding adds skill and examines and add MD5, transcoding and add MD5, transmission and add that transcoding, transmission add MD5, transmission adds transcoding and adds MD5; To the task that mix multiple subtask, the one-level that system carries out task splits, and is split as the subtask of a plurality of single task role types automatically;
The step that task scheduling unit task scheduling begins: after being used for the success initialization of task scheduling unit, the beginning duty cycle;
Detect the step whether idle task executing units is arranged: be used for the task scheduling unit and detect whether idle task executing units is arranged; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time; If " be " then carry out next step, being not only that whole task executing units is idle in this step just thinks the free time when the task of task executing units is full inadequately, to think that also this task executing units is idle;
The step whether waiting task is arranged: be used for the task scheduling unit and detect whether waiting task is arranged; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step;
The balanced step that detects of path load: be used for the task scheduling unit task can be detected in employable each path, the parameter of detection comprises: path priority level, speed-limiting messages;
Judge whether the step in operable path: be used for the task scheduling unit and use the requirement in path and the operating position of existing route to compare to task; Judged whether operable path; If " deny " then get into resting state; " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if " being " then carry out next step;
Task is carried out the step that secondary splits: be used for the task scheduling unit secondary fractionation is carried out in task or subtask that the one-level task splits the single task role type that obtains; Comprise: FTP splits, middle route splits, distributed fractionation, so that according to the actual performance element and the path of the state assignment task of path and task executing units;
The step of coupling path and task executing units: be used for the access ability of task scheduling unit, select to be fit to carry out path and task executing units through the various piece that splits task according to task executing units and path;
The step of Processing tasks: be used for task executing units task is handled, in the task implementation, according to the progress report time interval and the percentage-proportion intervals of configuration, to task scheduling unit Report Tasks progress;
The step of updating task state: be used for the information that the task scheduling unit receives that the Task Progress report updating task of task executing units is carried out;
Task is carried out the step of end and result's readjustment: complete back to task scheduling unit Report Tasks execution result in the task executing units task; The task scheduling unit is through the task result readjustment service of application and interface server calling task submission side, readjustment task action result;
Judge whether the step that manually stops: be used to judge whether manually stop the task scheduling circulation, if " denying " then get back to " step that task scheduling unit task scheduling begins ", if " being " then stop task scheduling;
Query task result's step: be used for the task side of submission to calling application and interface server; Query task implementation progress and execution result; This step and " judge whether manually stop step " parallel execution, and can carry out in each process in " step of Processing tasks " and carry out.
2. method according to claim 1 is characterized in that, described task scheduling unit and a plurality of task executing units are separately positioned in the server separately, forms task scheduling server and a plurality of task and carries out server.
3. method according to claim 2 is characterized in that, described task scheduling server has subsequent use task scheduling server, is connected through heartbeat between described task scheduling server and the subsequent use task scheduling server.
4. method according to claim 1 is characterized in that, described task scheduling unit is embedded in the task executing units.
5. method according to claim 1 is characterized in that, said that task is carried out the substep that step that secondary splits comprises is following:
Need to judge whether the substep of FTP fractionation: be used for judging whether the transcoding task has the FTP path; And determine whether needs carry out FTP and split according to looking audio format configuration; If " be " then get into " substep that FTP splits ", if " deny " then get into " substep of route fractionation in the middle of whether needing ";
The substep that FTP splits: be used to utilize buffer storage; Whether according to source or target is the FTP path, is split as 2 or 3 subtasks to a FTP transcoding task: FTP is to middle cached copies from the source, middle buffer memory transcoding; Copy to target FTP behind the transcoding, simultaneously the updating task tree structure;
The substep whether in the middle of needing route splits: be used to check the task scheduling unit whether ability while access originator and destination path are arranged; If do not have the task scheduling unit of simultaneously access originator and destination path then get into " substep that middle route splits ", if route does not split then entering " whether the distributed fractionation substep of needs " in the middle of not needing;
The substep that middle route splits: utilize buffer storage, be split as two subtasks to a task: source path is to the subtask of buffer storage, and buffer storage is to the subtask of destination path;
The substep that whether needs distributed fractionation: for the transcoding task; Option, task priority, the task executing units quantity factor of current free time during according to the task of submission to; Whether decision carries out distributed fractionation; If " be " then get into " substep of distributed fractionation ", if " deny " then get into " step of mating path and task executing units ";
The substep of distributed fractionation:, tear a transcoding task for a plurality of transcodings subtask open and to add an automatic merging task according to the quantity of present idle task performance element; There is a fragment sequence number each transcoding subtask; Be responsible for generating a fragment of file destination,, merge task automatically by the concurrent execution of a plurality of task executing units; Wherein, in time merge into the final objective file to the fragment of transcoding completion according to the fragment numeric order.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009103107033A CN101707543B (en) | 2009-11-30 | 2009-11-30 | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009103107033A CN101707543B (en) | 2009-11-30 | 2009-11-30 | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101707543A CN101707543A (en) | 2010-05-12 |
| CN101707543B true CN101707543B (en) | 2012-03-28 |
Family
ID=42377738
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009103107033A Expired - Fee Related CN101707543B (en) | 2009-11-30 | 2009-11-30 | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101707543B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102137276B (en) * | 2010-01-27 | 2013-08-14 | 新奥特（北京）视频技术有限公司 | Check system with interactive function |
| CN102438030A (en) * | 2010-09-29 | 2012-05-02 | 新奥特（北京）视频技术有限公司 | Packaging system and method suitable for broadcast and television cloud architecture platform |
| CN102006499B (en) * | 2010-12-10 | 2012-10-24 | 北京中科大洋科技发展股份有限公司 | Method for detecting video and audio quality of digital television program file |
| CN102546270A (en) * | 2010-12-13 | 2012-07-04 | 深圳市财付通科技有限公司 | Network system control method and device utilizing same |
| CN102655518B (en) * | 2011-03-03 | 2016-03-02 | 哈尔滨工业大学 | Transaction processing system and method |
| JP2012221155A (en) * | 2011-04-07 | 2012-11-12 | Sony Corp | Reproduction device and reproduction method |
| CN102752497A (en) * | 2011-12-27 | 2012-10-24 | 新奥特（北京）视频技术有限公司 | Comprehensive programming managing platform based on programming |
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN103853607B (en) * | 2012-12-03 | 2017-06-23 | 中国银联股份有限公司 | A kind of mutual Preparation Method of task scheduling |
| CN103167222B (en) * | 2013-03-29 | 2016-06-08 | 北京中科大洋科技发展股份有限公司 | A kind of nonlinear cloud editing system |
| CN108255820B (en) * | 2016-12-28 | 2022-03-04 | 阿里巴巴集团控股有限公司 | Method, device and electronic device for data storage in distributed system |
| CN107844368A (en) * | 2017-10-26 | 2018-03-27 | 苏州市伏泰信息科技股份有限公司 | Environmental sanitation platform task scheduling system |
| CN107908487B (en) * | 2017-11-08 | 2020-11-24 | 中国平安人寿保险股份有限公司 | Task control management method, device and equipment and computer readable storage medium |
| CN107885589B (en) * | 2017-11-22 | 2021-02-12 | 贝壳找房(北京)科技有限公司 | Job scheduling method and device |
| CN108289226B (en) * | 2018-01-19 | 2021-02-12 | 数码辰星科技发展（北京）有限公司 | Method, server and system for showing digital movie video data |
| CN108647093A (en) * | 2018-05-09 | 2018-10-12 | 厦门南讯软件科技有限公司 | A kind of distributed task scheduling processing system and its application method |
| CN108900639B (en) * | 2018-08-10 | 2021-04-06 | 乾元云硕科技(深圳)有限公司 | Cluster type cloud computing system |
| CN111338917B (en) * | 2018-12-19 | 2023-03-28 | 贵州白山云科技股份有限公司 | Dynamic control method and device for determining server service capability |
| CN114079749A (en) * | 2020-08-18 | 2022-02-22 | 山东汉鑫科技股份有限公司 | A cross-platform system for the field of manufacturing intelligent production business |
| CN111935467A (en) * | 2020-08-31 | 2020-11-13 | 南昌富佑多科技有限公司 | Outer projection arrangement of virtual reality education and teaching |
| CN115604504B (en) * | 2021-07-09 | 2026-02-03 | 上海哔哩哔哩科技有限公司 | Video DAG-based video processing method and manuscript processing method |
| CN114168601B (en) * | 2022-02-10 | 2022-07-22 | 浙江大学 | Mobile-end-oriented data stream processing method and equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1760900A (en) * | 2004-10-15 | 2006-04-19 | 中央电视台 | Broadcast television media asset management system and regulation and control method thereof |
-
2009
- 2009-11-30 CN CN2009103107033A patent/CN101707543B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1760900A (en) * | 2004-10-15 | 2006-04-19 | 中央电视台 | Broadcast television media asset management system and regulation and control method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 顾军等.媒体数据交换平台相关技术的探讨.《现代电视技术》.2009, * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101707543A (en) | 2010-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101707543B (en) | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type | |
| JP7724320B2 (en) | Cloud Services for Cross-Cloud Operations | |
| US8788565B2 (en) | Dynamic and distributed queueing and processing system | |
| CN112882813B (en) | Task scheduling method, device and system and electronic equipment | |
| US7743142B2 (en) | Verifying resource functionality before use by a grid job submitted to a grid environment | |
| US7676516B2 (en) | System and method for the optimization of database access in data base networks | |
| TWI224899B (en) | Dynamic binding and fail-over of comparable web service instances in a services grid | |
| EP2248311B1 (en) | Method and system for message delivery in messaging networks | |
| CN109815294B (en) | Distributed parallel data storage method and system without main nodes | |
| US8832215B2 (en) | Load-balancing in replication engine of directory server | |
| US10747584B2 (en) | Security-aware partitioning of processes | |
| US10333792B2 (en) | Modular controller in software-defined networking environment and operating method thereof | |
| US7739420B2 (en) | Communication error information output method, communication error information output device and recording medium therefor | |
| CN117041893B (en) | Method and system for sending international short message by using cloud computing technology | |
| CN102868594B (en) | Method and device for message processing | |
| US12088347B2 (en) | Intelligent monitoring and repair of network services using log feeds provided over Li-Fi networks | |
| CN111294383A (en) | IoT Service Management System | |
| CN114301982B (en) | A tenant- and gateway-based IoT device terminal access system and method | |
| CN119766817B (en) | New energy automobile data forwarding method based on distributed architecture | |
| US11775401B1 (en) | Intelligent coordination of log analysis and repair processes in a multi-cloud system | |
| KR20250165046A (en) | Method for managing interface, interface governance system, and a storage medium storing a computer-executable program to manage the interface | |
| CN119130076A (en) | Value-added service processing system, method and storage medium | |
| CN116708550A (en) | A message push method, device, equipment and storage medium | |
| CN118413440A (en) | A node control method, device and equipment | |
| WO2023032103A1 (en) | Job control system and job control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120328 Termination date: 20211130 |