Why did Zambeel fail

DE60121930T2 - Method for streaming a single media track to multiple clients - Google Patents

Method of streaming a single media track to multiple clients Download PDF

info

Publication number
DE60121930T2
DE60121930T2DE60121930TDE60121930TDE60121930T2DE 60121930 T2DE60121930 T2DE 60121930T2DE 60121930 TDE60121930 TDE 60121930TDE 60121930 TDE60121930 TDE 60121930TDE 60121930 T2DE60121930 T22DE 60121930T
Authority
DE
Germany
Prior art keywords
media
track
metadata
file
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
DE60121930T
Other languages
English (en)
Other versions
DE60121930D1 (de
Inventor
Geetha Palo Alto SRIKANTAN
Aravind New York NARASIMHAN
Seth Concord PROCTOR
Jan San Francisco BRITTENSON
Matthew San Jose SHAFER
Jonathan Fremont SERGENT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Microsystems Inc
Original assignee
Sun Microsystems Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US19575500PpriorityCritical
Priority to US195755Ppriority
Application filed by Sun Microsystems IncfiledCriticalSun Microsystems Inc
Priority to PCT / US2001 / 011137 priority patent / WO2001077870A2 / en
Publication of DE60121930D1publicationCriticalpatent / DE60121930D1 / de
Application granted granted Critical
Publication of DE60121930T2publicationCriticalpatent / DE60121930T2 / de
Anticipated expirationlegal-statusCritical
Expired - Lifetimelegal-statusCriticalCurrent

Left

  • 230000004044responseEffects0.000claimsdescription2
  • 238000000034methodMethods0.000description18
  • 230000005540biological transmissionEffects0.000description16
  • 230000000875correspondingEffects0.000description9
  • 239000000284extractSubstances0.000description6
  • 238000004891communicationMethods0.000description4
  • UIIMBOGNXHQVGW-UHFFFAOYSA-MbufferSubstances [Na +]. OC ([O -]) = OUIIMBOGNXHQVGW-UHFFFAOYSA-M0.000description3
  • 150000001875compoundsChemical class0.000description3
  • 239000000969carrierSubstances0.000description2
  • 238000010586diagramMethods0.000description2
  • 239000000203mixtureSubstances0.000description2
  • 210000001138TearsAnatomy0.000description1
  • 239000002131composite materialSubstances0.000description1
  • 238000007906 compressionMethods0.000description1
  • 230000001276 controlling effectEffects0.000description1
  • 238000005516 engineering processMethods0.000description1
  • 230000000977initiatoryEffects0.000description1
  • 230000003287opticalEffects0.000description1
  • 230000001360synchronisedEffects0.000description1
  • 230000002123temporal effectEffects0.000description1

Classifications

    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L29 / 00 — Arrangements, apparatus, circuits or systems, not covered by a single one of groups H04L1 / 00 - H04L27 / 00
    • H04L29 / 02 Communication Control; Communication processing
    • H04L29 / 06 Communication Control; Communication processing characterized by a protocol
    • H04L29 / 0602 — Protocols characterized by their application
    • H04L29 / 06027 — Protocols for multimedia communication
    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65 / 00 — Network arrangements or protocols for real-time communications
    • H04L65 / 10 — Signaling, control or architecture
    • H04L65 / 1066 — Session control
    • H04L65 / 1069 — Setup
    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65 / 00 — Network arrangements or protocols for real-time communications
    • H04L65 / 40 — Services or applications
    • H04L65 / 4069 — Services related to one way streaming
    • H04L65 / 4076 — Multicast or broadcast
    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65 / 00 — Network arrangements or protocols for real-time communications
    • H04L65 / 60 — Media handling, encoding, streaming or conversion
    • H04L65 / 608-Streaming protocols, e.g. RTP or RTCP
    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65 / 00 — Network arrangements or protocols for real-time communications
    • H04L65 / 80 — QoS aspects
    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67 / 00 — Network-specific arrangements or communication protocols supporting networked applications
    • H04L67 / 28 — Network-specific arrangements or communication protocols supporting networked applications for the provision of proxy services, e.g. intermediate processing or storage in the network
    • H04L67 / 2819 — Enhancement of application control based on intercepted application data
    • H-ELECTRICITY
    • H04 — ELECTRIC COMMUNICATION TECHNIQUE
    • H04L-TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69 / 00 — Application independent communication protocol aspects or techniques in packet data networks
    • H04L69 / 30 — Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69 / 32 — High level architectural aspects of 7-layer open systems interconnection [OSI] type protocol stacks
    • H04L69 / 322 — Aspects of intra-layer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69 / 329 — Aspects of intra-layer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer, i.e. layer seven

Description

  • This invention relates to the field of computer systems. More specifically, in a media flow system in which a media track has associated metadata providing time, offset and other information, an apparatus and method is provided for streaming a media track to multiple clients with only one copy of the metadata Tracks.
  • Media streaming systems are designed to broadcast media programs and events, which may be pre-recorded or live, to client devices (e.g. computers, media players). A client can play a media stream as it receives it (i.e., before the stream or transmission is complete), allowing for faster or even real-time enjoyment of the media program or event.
  • Media can be transmitted in unicast or multicast mode. In unicast mode, the flow server maintains a reserved connection to each receiving client device, which gives the user extensive control over his or her electricity. For example, he or she may be able to stop a stream or rewind or fast forward through the flowing medium or perform other control functions. However, this can result in inefficient use of bandwidth for a large number of users. In multicast mode, the media stream server broadcasts a program to multiple users at the same time, which uses less bandwidth. This type of data flow is thus comparable to traditional transmission and users have little control over their individual flows. Live events can usually be broadcast in multicast mode as it is more efficient for serving large numbers of users. And since it is a live event that is enjoyed in real time, there is little need to manipulate the streaming media.
  • IBM Technical Disclosure Bulletin V40 / 10, October 97, pages 123-127 describes the use of structured metadata for application specific viewers for broadcast Internet video / audio. The navigation and selection for video and audio is accomplished using web browser technology. Metadata is returned from an application server to a client via an http server and causes a video / audio viewer to be started on the client. Information in the metadata includes the address of the application server and the type of encoding of the video / audio.
  • Streamed media are often composed of multiple tracks, pieces or files. For example, an audiovisual program generally includes at least one audio track and at least one video track. Each track includes media of the appropriate type (e.g., audio, video) and may also include metadata that is used to properly broadcast the media. A track's metadata can include information for identifying a media segment or sample (or other media unit) that should be played in a given time index within the program, for determining where the segment or sample is located in the file, etc. The metadata of a media track can thus be used by a media flow server to transmit the media track in the correct order with the appropriate timing and so on.
  • When a media track or program is broadcast to multiple clients, existing systems typically store separate copies of each track's metadata for each client. More precisely, these systems repeatedly extract the media data from a media file that contains the medium to be transmitted, each time a new client requests the media, and store it. Serving a large number of clients can consume a significant amount of server resources (e.g. memory, processor, disk usage) and limit the number of clients the server can support.
  • Some existing systems also typically assign only one file descriptor, which is shared by all clients receiving a particular media stream, for accessing the media file containing the transferred media. This can lead to significant competition between the client streams, as each tries to search (i.e. find) and extract a different media segment or sample.
  • Furthermore, when broadcasting a multi-track media program to a client, existing media streaming systems attempt to maintain synchronization between tracks so that the appropriate corresponding media track is broadcast or played for each unit of time of the program, but they may not be able to synchronize to regain when it is lost. For example, if the media for one track is being received (e.g. from a storage device) at a slower rate than the media for another track, an existing media streaming system can simply continue to transfer the media even if the tracks increase got out of sync. Other systems can simply pause a media stream if synchronization is lost without attempting to correct the situation.
  • An apparatus, method and computer readable storage medium are provided in accordance with the present invention and as defined in the claims for sharing a copy of the metadata of a media track for broadcasting the media track to multiple clients.
  • In one embodiment, a file track is created to store the metadata of a single media track. For a media program that has multiple tracks, a separate file track is created for each track. Separate file track handlers are established for each client media stream. Each file track handler acts as an interface between its client stream - different client streams can be within the medium at different time indices - and the single instance of the media metadata.
  • Figure 13 is a block diagram illustrating a server configured to transfer media in accordance with an embodiment of the present invention.
  • Figure 13 is a block diagram illustrating the use of a single copy of the media track metadata to deliver the media to multiple clients in accordance with an embodiment of the invention.
  • Figure 10 illustrates a configuration of program objects that cooperate to deliver a media track to multiple clients with a single copy of the track's metadata in accordance with an embodiment of the present invention.
  • - comprise a flow diagram illustrating a method for broadcasting a media track to multiple clients with a single copy of the track's metadata in accordance with an embodiment of the present invention.
  • Figure 11 illustrates a configuration of program objects for resynchronizing media within a media stream, in accordance with an embodiment of the present invention.
  • Figure 13 is a flow diagram illustrating a method for re-synchronizing media within a media stream, in accordance with an embodiment of the present invention.
  • DETAILED DESCRIPTION
  • The programming environment in which a present embodiment of the invention is carried out illustratively includes a general purpose computer or special device such as a computer. A computer server configured to provide data or media streaming services to computer or communication facilities of virtually any configuration (e.g., wired, wireless, portable, tabletop). Details of such computers and other devices (e.g., processor, memory, data storage, display) are known and may be omitted for clarity. Furthermore, embodiments of the invention are described and they can be implemented in an object-oriented programming environment. Suitable variations of embodiments can be implemented using other programming models or frameworks that would be apparent to those skilled in the art.
  • It will also be understood that the techniques of the present invention could be implemented using a variety of technologies. For example, the methods described herein can be implemented in software running on a computer system, or they could be implemented in hardware using either a combination of microprocessors or other specially designed application specific circuitry, programmable logic devices, or various combinations thereof. In particular, the methods described here can be carried out by a series of computer-executable commands that are stored on a storage medium, such as. A carrier wave, disk drive, or computer readable medium. Exemplary forms of carrier waves can take the forms of electrical, electromagnetic, or optical signals that carry digital data streams along a local area network or a publicly accessible network such as a public address network. B. the Internet, forward.
  • In one embodiment of the invention, a media or data stream server is configured to transmit media or data to multiple clients. A media program or event broadcast to multiple clients includes one or more tracks (e.g. audio, video), each of which can be stored in one or more media files. When an initial request for the media program is received, the track's metadata is retrieved and placed in a file track. The metadata can contain information relating to: timing (e.g. to synchronize the media of the track with other program tracks), positions of media segments or samples (or other media units) within a media file, hierarchies of media units (e.g. pieces, samples that have a piece), edit tables, etc.
  • For each client to which the media track is transmitted, separate file track handlers are set up in order to reference or refer to metadata within the file track. Instead of storing separate copies of the track metadata for each client stream, each file track handler of the client references (e.g. with pointers) to the individual file track in order to call up the clock point, to locate media segments in a media file, etc. Furthermore, each file track handler is assigned to a file descriptor in the track's media files for finding and getting media.
  • Specific embodiments of the invention are described below, as well as being used for transferring QuickTime media from a UNIX-based computer system, such as a computer. B. a system that uses the Solaristm Operating system run by Sun Microsystems, Inc., can be implemented. Such embodiments can be modified for use with other types of media and other computer systems, as will be apparent from the following description.
  • ILLUSTRATIVE MEDIA POWER SERVER
  • A media stream allows a user to receive and enjoy media content without having to wait for the entire program or presentation to be downloaded to his or her client device. For example, a user can enjoy a pre-recorded program or experience a live event in real time without waiting for the entire program to be received.
  • A media stream server according to a present embodiment of the invention can operate in a "reflection mode of operation" in which the server receives a media stream from another stream system or server (usually in multicast mode) and delivers the media to one or more users (in unicast or multicast mode) ) forwards.
  • The streaming of real-time media places barriers on the issuing server because the delivery of each frame or other unit of the media must be performed in a specific order and within a certain period of time. Thus, regardless of the number of clients being served, a media stream server must strive to meet real-time media streaming requirements so that the quality of service for the users does not fall to an unacceptable level. For example, regardless of the type of program (e.g., live or pre-recorded) and streaming mode (e.g., unicast or multicast), media to be transmitted is generally compressed to reduce the bandwidth it consumes in transit, which helps to ensure timely delivery of the media to a client.
  • A media stream server according to an embodiment of the invention is configured to serve QuickTime media and / or other forms of media in a unicast or mullticast mode over a proprietary or publicly accessible network, such as e.g. B. to transmit the Internet. Media streams are formatted according to a set of protocols compatible with the transmission medium, especially if QuickTime media is being transmitted, the server can be configured for RTSP (Real-Time Streaming Protocol) to facilitate control of the media flow by a client Be configured for RTP (Real-Time Transport Protocol) to deliver the stream to the client and / or receive media from another source, be configured for RTCP (Real-Time Transport Control Protocol) to provide information regarding the stream quality to receive or exchange, be configured for SDP (Session Description Protocol) to describe the media to the client, etc. Other embodiments may be configured for other media protocols.
  • Figure 3 illustrates a media stream server configured to broadcast QuickTime media in accordance with an embodiment of the invention. In the media stream server serves the clients,. The medium transmitted to the clients may include a prerecorded program that is retrieved from the storage device, or it may be a real-time program that is received by the server (e.g., as part of a multicast broadcast). The media stream server can thus transmit live events (e.g. concerts, news programs, sporting events), films, documentaries, training videos, teaching programs, etc.
  • The media transfer can require multiple connections between the media stream server and a client. In the embodiment illustrated in Figure 4, a first connection for RTSP (e.g. the connection, the connection) is provided to enable a client to control a media stream. In particular, a client uses an RTSP connection to send commands to the media stream server. The media stream commands that a client can transmit to the server in this embodiment include commands such as: B. Options to get a list of supported commands, Describe to describe the media program to the server, Setup to identify the desired tracks to be received (each track can have a different media form, e.g. . Video, audio, etc.), play to play a media track or program, pause to temporarily stop the transmission, teardown to end a stream, etc. Thus the client can, for example, establish an RTSP connection with the server and issue the Describe command to receive a description of the content and tracks available for transmission. The client can then transmit a setup request for one or more tracks.
  • When a client issues a setup command to the server, the server establishes an RTP connection (e.g. connection, connection) and an RTCP connection (e.g. connection, connection) for the selected track (s) ( s). When the play command is received, the server starts transmitting the media packets to the client over the RTP connection. And the server and the client can exchange RTCP packets over the RTCP connection, which describe the quality of the stream. When a teardown command is issued, the server closes the affected flow connections with the issuing client.
  • The various connections employed by the media stream server can use Transport Control Protocol (TCP) sockets for a compatible communication medium over which the server and a client communicate (e.g., the Internet). In other embodiments of the invention, the sockets can be configured according to a different protocol (e.g. http-HyperText Transport Protocol, FTP-File Transfer Protocol).
  • As already described, the media stream server can transmit real-time or live media to clients and can also transmit pre-recorded media. Furthermore, in the reflection mode of operation, the media stream server can receive media that it has received from another entity, e.g. B. receives a live event, a video camera, a broadcast from another server (e.g. server), etc., forward to the clients. In this situation, the media stream server acts as a client and receives media packets over an RTP connection established with the entity.
  • The clients are equipped with suitable media players for playing the media that have been transferred from the media stream server. For QuickTime media streaming, clients can use a QuickTime player such as B. from Apple Computer, Inc. available. The client computing devices can operate on almost any operating system for which a suitable media player is available (e.g. Solaris, Mac OS, Windows, Linux). Because client devices have relatively low bandwidth communication capabilities (e.g., 56K modems), media streams can be sent at relatively low bit rates. Higher bit rates can of course be implemented for more stable clients. Clients can identify media that are being transmitted to them by providing a URL (Uniform Resource Locator), a file name, a program name (e.g. film name, song title), etc.
  • U.S. Patent Application No. 20010029548, filed April 6, 2001, entitled "Method and Apparatus for Handling Events Received at a Server Socket" describes a media stream server configuration for transferring media to multiple clients over a limited number of communication sockets.
  • STREAM MEDIA TO MULTIPLE CLIENTS WITH A SINGLE FILE RACK
  • In one embodiment of the invention, a media stream server transmits media accompanied by metadata to facilitate the streaming process. For example, metadata for a QuickTime media track can indicate which unit and which piece of the medium (e.g. audio sample, film segment) corresponds to a given time index within the program of the medium and where it can be found within a media file. In this embodiment, the metadata can be assembled once (e.g., extracted from one or more media files for a given media track), but can be used to facilitate broadcasting of that track to many clients. This scheme is thus more efficient than existing transmission systems in which separate copies of the metadata are assembled for each individual client stream.
  • The media data, as soon as it has been extracted using the metadata, is placed in RTP packets and transmitted to the clients. The media data can be prepared for transmission in almost any format that can be handled by a client. In particular, the media can be placed in packets without changing their encoding, compression, or other attributes.
  • In clear terms, a client can request a media stream that has multiple tracks or can request specific tracks individually. Each track has its own metadata and can be sent to a client as a separate stream (i.e. multiple tracks may or may not be combined into a single stream). As described above, in one embodiment of the invention the media is sent in RTP packets, in which case the transmission can be controlled by a client via RTSP commands and the flow quality can be reported via RTCP. Other suitable protocols are used in other embodiments of the invention.
  • When the media stream server operates in a reflection mode, in which it receives media (as RTP packets) from a source and transmits them to clients, metadata for the media can be received with or before the media.
  • illustrates the use of a single file track and multiple file track handlers to deliver a media track to multiple clients without separately extracting or making separate copies of the track's metadata, according to an embodiment of the invention.
  • In the illustrated embodiment, the media program includes an audio track and a video track. Each track includes its displayed media type for the program (e.g. audio, video) and metadata that includes: identify which media unit (e.g. audio sample, video frame) corresponds to a given time index in the media program, identify which piece (e.g. a collection of media units) a given unit belongs to or is included in, lists the media units within a given track, identifies where a given track or media unit is stored within the media program file, includes edit tables that allow content tools to insert or delete portions of the track without copying or compacting the entire track (e.g., they allow fragmenting a track), etc. Illustratively, the audio track and video track are saved as separate files (or sets of files). Alternatively, however, different types of tracks or sections of different track types can be stored in a single file.
  • The metadata of a media track can be stored contiguously within the file (s) containing the media of the track, can be distributed over the media file (s) of the track or can even be stored as a separate track.
  • In accordance with this embodiment of the invention, the metadata of each track is extracted and stored in separate file tracks. The metadata for the audio track is thus composed as a file track, while the metadata for the video track is stored in the file track. The different types of metadata within a track can be stored as separate tables, lists or other data structures.
  • Separate file track handlers are established for each client to which a track of the media program is transmitted. Thus, the file track handler is set up for the client to access the metadata of the file track or. Similarly, for the client, the metadata stored in the file track for the audio track is accessed by the file track handler, while the file track metadata for the video track is accessed via the file track handler. Clearly, each file track handler includes one or more pointers or other references to the associated file track with which the various types of information are accessed.
  • In addition, each file track handler or set of file track handlers for a given client stream can be assigned a separate file descriptor for accessing the media file (s) that store the track's media. Thus, in the embodiment of Figure 8 in which the audio track and video track are stored in separate files, each file track handler,,, are assigned their own file descriptor for locating and extracting media for transmission. Alternatively, a single file descriptor can be shared by all of the file track handlers associated with a single media track to access the file that stores the track's media.
  • While the media is being transferred to a client (e.g. after the client requests a track or program to be played), the file track handlers for the client's stream access the file tracks to determine the correct sequence of the media (i.e. the media unit ( en) to be transmitted for each time index of the media). With this information, the file track handlers extract the media from the media file (s) so that they can be placed in RTP packets (or packets configured according to some other suitable protocol) and to the client over an RTP connection can be transferred. When a client terminates its stream, the file track handlers are deleted.
  • In one embodiment of the invention, a file track is configured to monitor the number of file track handlers that referenced it. In this embodiment, when all inflows of the affected media track have ended, the file track can remove itself and thereby free up additional resources for other processes or media tracks.
  • illustrates various program objects used in an embodiment of the invention. Appropriate corresponding program modules, which are configured to operate in a manner similar to the illustrated program objects, can be generated for computer systems with non-object-oriented programming methods.
  • In this embodiment, the track represents a class of objects configured to compile metadata for a media track and allow multiple file track handlers to use the metadata for different client streams. Several types of track objects can be created from the track, such as: B. the live track for assembling metadata for broadcasting live or real-time event media, and the file track for assembling metadata for stored or prerecorded media. In an alternative embodiment of the invention, a type of track object (e.g. the file track) can be configured to facilitate the transmission of both prerecorded and live media. In yet another alternative configuration, the live media do not have to be in the form of tracks and can therefore be made available to clients in any other way than described here.
  • The QuickTime Filetrack and the MPEG Flietrack are filetrack objects configured to handle metadata for two specific forms of media (e.g. QuickTime and MPEG). Objects created from the track can adopt attributes and methods that are useful for handling live media, prerecorded media, and / or other media. Among the inherited attributes may be, for example, one that keeps a count of track handlers referring to a given track.
  • The track handler represents a class of objects configured to facilitate the transfer of track media to a client using a shared collection of track metadata. Clearly speaking, the live track handle (live track handler) represents a class of track handle objects that are configured for the transmission of live media, while the file track handle represents a class of track handle objects that are configured for previously recorded or other stored media. In an alternative embodiment of the invention, one type of track handle object (e.g., the file track handle) can be configured for the transmission of both prerecorded and live media. According to another alternative live configuration, the live media do not have to take the form of a track and can therefore be delivered to the clients in any other way than described here.
  • Different types of filetrack handle objects for specific media forms can be realized from the filetrack handle, e.g. B. a QuickTime file track handle for QucikTime media and an MPEG file track handle for MPEG media.
  • In this embodiment of the invention, each track handle object derived from the track handle includes appropriate methods for moving to a particular time index in a media program or track, reading media data in a buffer (for transmission to a client), etc. Thus, the track handle objects hold the status information regarding the current playback position of a client in a media track and can include one or more buffers for sending media packets to clients and one or more pointers or references to their corresponding track objects for accessing the metadata.
  • Once a file track object is established for a given media track, subsequent client requests for that track use the established file track instead of reassembling the track's metadata. Thus, file track objects can be configured to implement new file track handle objects for new client streams.
  • FIG. 10 illustrates a method for providing media streams from a track to multiple clients while only maintaining a copy of the media's metadata in accordance with an embodiment of the invention.In this process, the same media can be transmitted to each client, but with a different frequency. That is, different client streams can transmit media from different time indices within the media track at any given time.
  • In the state of the illustrated method, a request to transmit pre-recorded media that is not already being transmitted is received. The request can be received, for example, as an RTSP command to set and play a particular media track. Other forms of requesting media can be used for other protocols or forms of media.
  • In the state, the media stream server creates a track object in order to manage the shared access to the requested metadata. The illustrated method shows the creation of a file track object which, as described above, can be configured for prerecorded media programs and tracks.
  • In the state, the file track object extracts the metadata from the media track and saves it. The file track object may have to analyze one or more files containing the media track in order to compose the metadata.
  • In the state, a file track handle is generated for the new client stream in order to provide a mapping or an interface between the stream-specific client context and the track-specific metadata held by the file track object. The filetrack handle object can be provided with one or more references to the composite metadata (e.g. different references for different tables or different types of metadata). In addition, a file descriptor or similar resource is assigned to the filetrack handle to facilitate its access to the file (s) containing the media (e.g. to find and extract media to be transferred).
  • In the state, the requested media are transferred to the new client. The client can issue commands to control the stream - e.g. B. to fast rewind or fast forward to locate a certain part of the media, to pause the transmission, etc.
  • In the state, while the media is being transmitted to one or more clients, the media flow server is alerted to each new request for the same media. When a new request is received, the illustrated method returns to the state to set a new file track handle object and stream. Otherwise the procedure continues with state.
  • In the state, the media stream server determines whether all client streams for the medium have ended. The server can make this decision, for example whenever a client stream is terminated. Illustratively, when a client stream is terminated, its associated file track handle object is deleted. If any streams are still in use or being established, or if new streams are in the process of being established, the procedure returns to state. Otherwise the procedure continues with status.
  • In the state, after all client streams have ended, the file track object is deleted and the metadata is removed from memory.
  • RESYNCHRONIZATION OF MEDIA DURING THE TRANSMISSION
  • A media stream is considered asynchronous if media for one or more tracks in the program or event that is being broadcast is not streaming fast enough to keep up with the stream's time index. In other words, a current time index of the broadcast media program is the time index for which all tracks should transmit corresponding media. If the media for a particular track is lagging behind the current time index, or if no media is being transmitted for the track at all, then the stream can be considered asynchronous. Loss of synchronization can occur, for example, when a source of the track's media (e.g. a storage device, live media feed) cannot feed the media at a sufficiently high speed.
  • If synchronization is lost, a media stream server can attempt to re-synchronize the media stream by selecting a new, later media time index at which the transmission will resume. The server then requests or waits for media according to the new time index and, until the new time index is reached, the server can stop transmitting or can continue transmitting media that is currently being transmitted (e.g. for a track which is still synced).
  • When the new time index is reached, when the necessary media is received, the media stream server begins transmitting the appropriate media. However, if the media corresponding to the new time index is not available when the new time index is reached, the server will make another attempt to resynch to another later new time index and may allow more time to fetch the appropriate media . Attempts to resynchronize a media stream can be made a configurable number of times, and the server can terminate the stream if all attempts have failed.
  • represents various program objects that can work together to facilitate resynchronization of a media stream. Appropriate corresponding program modules configured to operate in a manner similar to the illustrated program objects can be built for computer systems using non-object oriented programming techniques.
  • In the TrackStream represents an object class that is configured to process data from a media track for transmission to a client. The LiveTrackStream is a TrackStream object class that is configured for broadcasting live media, while the FiletrackStream is a TrackStream object class that is configured for prerecorded or other media. A TrackStream object can contain interfaces for the retrieval of specific track media (e.g. for a specified time index). In the FiletrackStream, different types of FiletrackStream objects can be implemented or called for specific media forms, such as B. the QuickTime FiletrackStream for QuickTime media and the MPEG FiletrackStream for MPEG media.
  • The program objects shown in FIG. B. those illustrated and described in connection with. In particular, a FiletrackStream object can contain or refer to a Filetrackhandle object for finding specific locations in a media file, reading data from the file, etc.
  • The stream is an abstraction of a media stream to a client. Various types of media streams can be presented, for example, the live stream for a stream of live media and the file stream for a stream of prerecorded media. For different media protocols or forms, different types of stream objects can be created to handle the transmission. For example, FileRTPStream is configured to broadcast media to clients over RTP, while FileHTTPStream is configured to broadcast media over http.
  • In the illustrated system, each stream object includes methods to start, stop, pause, and otherwise control a media stream (e.g., in response to commands from a client). A stream object has one or more sockets through which it communicates with a client and interacts with or may even contain a trackstream object to retrieve media data for transmission. Clearly, and since a media stream can have multiple tracks, each stream object can be linked to or contain multiple track stream objects. For example, if FileRTPStream is created to broadcast a basic audiovisual media program consisting of an audio track and a video track to a client, FileRTPStream may interact with or include separate FiletrackStream objects for each track. As described below, when a track gets out of sync with a media stream, the stream object controlling the transmission can start a resynchronization procedure.
  • A client connects to a media stream server to retrieve a prerecorded media program and a stream object of the appropriate type or configuration, such as. B. FileRTPStream is generated to control the transmission of the media. FileRTPStream initiates or calls one or more trackstream objects of the appropriate type, such as B. the QuickTime file track stream. The file track stream object (s) creates or calls on a track or track handle objects of the appropriate type or configuration (e.g., as shown in Figure 4) to retrieve media from the track (s) of the program. The stream object controls the transmission (e.g. start, stop, interrupt) and resynchronizes the stream if necessary. The track stream object (s) controls the retrieval of the track media by initiating appropriate tasks and / or calling interfaces provided by the track or the track handle objects (e.g. to locate media data, data from a media file to read).
  • represents the resynchronization of a media stream. The media to be transmitted in the illustrated procedure has been pre-recorded and is therefore retrieved from one or more storage devices, but the procedure can be easily modified to receive live media from a live event or other server. In state, a prerecorded media program that has multiple tracks is transmitted to a client through an assigned stream object (e.g., FileRTPStream from).
  • In the state, a request is made or terminated (e.g. by the QuickTime file track stream from) to read media data for one of the program's tracks from a media file. The requested media data clearly correspond to an upcoming media time index (e.g. film time). While waiting for the requested data, the stream object can transmit media data that is already available (e.g. because it was received earlier), for the same or a different track, or it can wait until the requested data should be available.
  • In the state, the stream object determines whether the requested data is available, or it can be automatically woken up or otherwise alerted when the data is received. When the requested data has been received, the method returns to state to transmit the media data with the appropriate media time index. In particular, when media data for the tracks of the program is received in a temporal manner, the stream object can transmit the data with the corresponding media time index, thereby keeping the media program in sync. If the stream object in the state determines that the requested data is not available (e.g. a storage device is overloaded), which thus indicates a loss of synchronization, the procedure continues with state.
  • In the state, the stream object sets the current media time index up to a future time index and, in the state, queries media data that correspond to the future index. Thus, in this embodiment of the invention, the media stream server decides to drop or ignore data corresponding to the media time between the current and future time indexes. All such data (e.g. for a track that has not lost synchronization) can be clearly discarded. The stream object then goes to sleep or waits an appropriate amount of time for the requested media to be received. In state, the stream object determines whether the media data that corresponds to the new media time index has been received and is available for transmission. If so, the stream object returns to the state to transfer the media and get the next portion of the media.
  • However, if the media data is not received and is not available for transmission at the new time index, the stream object in the state determines whether the resynchronization has failed more than a limit number of attempts (e.g., 3). If not, the illustrated procedure returns to state to advance the media time index one more time. Every time the time index is advanced, it can be continued without a larger section. For example, the first time the time index is advanced by a period T, the next time it can be advanced by a period 2 * T, then 4 * T, 8 * T, and so on. If resynchronization fails repeatedly a predetermined number of times, the stream object halts the stream and the procedure ends.
  • It will be understood by those skilled in the art that the procedure illustrated in FIG. 4 is only one method of resynchronizing a media stream by dropping old media in favor of regaining resynchronization at a later time index.
  • The amount of media data requested or retrieved each time may be determined in part by the size or number of data buffers available, the frequency at which data requests are issued, the bit rate at which the media is transmitted to a client, etc.

Claims (16)

  1. A method for streaming media to multiple clients (12, 14), comprising: receiving a request to deliver media from a media track to a first client, extracting a set of metadata from the media track, the metadata being the identification and facilitate retrieval of the media from the media track, storing the extracted set of metadata in memory, streaming the media to a first client in a first stream using a first file track handler (12) to access the stored metadata, and Streaming the media to a second client in a second stream using a second file track handler () to access the stored metadata.
  2. The method of claim 1 further comprising: maintaining a first file descriptor assigned to the first file track handler for retrieving the media from the media track for the first stream, and maintaining a second file descriptor assigned to the second file track handler for retrieval the media from the media track for the second stream.
  3. The method of claim 1 or 2, further comprising: creating a file track (12) to place a copy of the stored metadata in memory and receiving a second request to flow the media to the second client prior to the flow of the media the first client is terminated, wherein the flow of media to the second client is performed in response to the second request.
  4. The method of claim 3, wherein flowing the media to the first client comprises: establishing a first set of references to the metadata, using the first set of references to identify a first portion of the media related to the first Allow clients to flow for a first time index and use the first set of references to locate the first section of media in the media track.
  5. The method of claim 4, wherein flowing the media to the second client comprises: establishing a second set of references to the metadata, using the second set of references to identify a second portion of media to flow to the second client for a second time index, and using the second set of references to locate the second portion of media in the media track.
  6. The method of claim 5, wherein the first set of references and the second set of references are used to access the metadata simultaneously.
  7. The method of any preceding claim, further comprising: removing the metadata from memory after the first stream and the second stream have ended.
  8. Method according to one of the preceding claims, wherein the media track is a track of a live event.
  9. Method according to one of Claims 1 to 7, in which the media track is a track of a pre-recorded media program.
  10. Computer readable storage medium that stores a program which, when executed by a computer, causes the computer to perform a method of flowing media to multiple clients as claimed in any preceding claim.
  11. An apparatus for streaming media to clients (12, 14) comprising: a first track of a media program stored on a first storage device, the first media track comprising: media and metadata configured to access to facilitate the media, a first memory, a file track (14) configured to store the metadata in the first shared memory, a set of file track handlers (14, b), each of the file track handlers configured to include Using the file track to access the stored metadata to facilitate the flow of the media to another client, and wherein the file track handlers access the stored metadata to identify portions of the media and locate the portions on the first storage device.
  12. The apparatus of claim 11, wherein each of the file track handlers is assigned a separate file descriptor for retrieving the media from the first storage device.